Hierarchical Self-Attention Hybrid Sparse Networks for Document Classification

Document classification is a fundamental problem in natural language processing. Deep learning has demonstrated great success this task. However, most existing models do not involve the sentence structure as text semantic feature architecture and pay less attention to contexting importance of words sentences. In paper, we present new model based on sparse recurrent neural network self-attention mechanism for document classification. Subsequently, analyze three variant GRU LSTM evaluating different datasets. Extensive experiments demonstrate that our obtains competitive performance outperforms previous models.